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Comparison  of  Several  Gradient  Algorithms 
For  Mathematical  Programming  Problems* 

by 

A.  MIELE2,  J.L,  TIETZE3,  and  A.V.  LEVY4 

Abstract.  In  this  paper,  the  numerical  solution  of  the  basic  problem  of  mathematical 
programming  Is  considered.  This  is  the  problem  of  minimizing  o  function  f(x) 
subject  to  a  constraint  afx)  =  0.  Here,  f  Is  a  scalar,  x  an  n-vector,  and  cr>a 
q -vector,  with  q  <  n. 

Six  variations  of  the  sequential  gradient- re storntion  algorithm  and  the 
combined  gradient -restoration  algorithm  are  considered,  and  their  relative 
efficiency  (in  terms  of  number  of  iterations  for  convergence)  is  evaluated. 

The  variations  being  considered  are  as  fellows: 

(I)  SGRA-CR,  sequential  gradient-restoration  algorithm,  complete  restoration, 

(II)  SGRA-IR,  sequential  gradient -restoration  algorithm,  incomplete  restoration, 

(ill)  SGRA-OR,  sequential  gradient -restoration  algorithm,  optional  restoration, 
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(iv)  CGRA-NR,  combined  gradient-restoration  algorithm,  no  restoration, 

(v)  CGRA-AR,  combined  gradient -restoration  algorithm,  alternate  restoration, 

(vi)  CGRA-OR,  combined  gradient -restoration  algorithm,  optional  restoration. 
Evaluation  of  these  algorithms  is  accomplished  through  eight  numerical 

examples.  The  first  two  examples  pertain  to  quadratic  functions  subject  to 
linear  constraints.  The  remaining  examples  pertain  to  nonquadratic  functions 
subject  to  nonlinear  constraints.  The  results  indicate  that  (a)  the  inclusion  of 
a  restoration  phase  is  necessary  for  rapid  convergence  and  (b)  the  algorithms 
with  alternate  restoration  or  optional  restoration  are  the  most  efficient  among 


those  considered  here. 
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1.  Introduction 

In  previous  papers  (Ref.  1  -3),  two  basic  algorithms  for  the  minimization 
of  constrained  functions  were  developed:  the  sequential  gradient -restoration 
algorithm  (SGRA)  and  the  combined  gradient-restoration  algorithm  (CGRA).  The 
former  is  an  iterative  algorithm  which  consists  of  the  alternate  succession  of 
gradient  phases  and  restoration  phases;  the  latter  is  an  iterative  algorithm  in 
which  the  gradient  phase  and  the  restoration  phase  are  combined  in  a  single 
phase. 

In  the  gradient  phase  of  SGRA,  one  generates  a  displacement  Ax  lowering 
the  value  of  the  function,  while  avoiding  excessive  constraint  violation;  in  the 

i 

restoration  phase  of  SGRA,  one  generates  a  displacement  Ax  restoring  the 
constrain!  to  a  predetermined  accuracy,  while  avoiding  excessive  change  in  the 
value  of  the  function.  On  the  other  hand,  in  the  gradient -restoration  phase  of 
CGRA,  one  generates  a  displacement  Ax  lowering  the  value  of  the  augmented 
function,  while  simultaneously  reducing  the  constraint  violation. 

In  this  paper,  six  variations  of  the  sequential  gradient -restoration 
algorithm  and  the  combined  gradient-restoration  algorithm  are  considered,  and 
their  relative  efficiency  (in  terms  of  number  of  iterations  for  convergence)  is 
evaluated  through  eight  numerical  examples.  The  variations  being  considered 
are  indicated  below: 

(i)  SGRA-CR,  sequential  gradient-restoration  algorithm,  complete  restoration, 

(ii)  SGRA-IR,  sequential  gradient -restoration  algorithm,  incomplete  restoration, 

(iii)  SGRA -OR,  sequential  gradient -restoration  algorithm,  optional  restoration, 

(iv)  CGRA -NR,  combined  gradient-restoration  algorithm,  no  restoration. 
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(v)  CGRA-AR,  combined  gradient -restoration  algorithm,  alternate  restoration, 

(vi)  CGRA-OR,  combined  gradient-restoration  algorithm,  optional  restoration. 


5 


AAR-94 


2.  Statement  of  the  Problem 

We  consider  the  problem  of  minimizing  the  function 

f=f(x)  (1) 

subject  to  the  constraint 

eo(x)  =  0  (2) 

where  f  is  a  scalar,  x  an  n-vector,  and  cpa  q-vector,  with  q  <n.  Here,  all  vectors 
are  column  vectors.  It  is  assumed  that  the  first  and  second  partial  derivatives 
of  the  function  f(x)  and  cp(x)  exist  and  are  continuous  and  that  the  constrained 
minimum  exists. 

2. 1.  First-Order  Conditions.  From  theory  of  maxima  and  minima, 
it  is  known  that  the  above  problem  is  equivalent  to  that  of  minimizing  the 
augmented  function 

F(x,X)  =  f(x)+XTcp(x>  (3) 

subject  to  the  constraint  (2).  Here,  the  q-vector  X  is  the  Lagrange  multiplier 
and  the  superscript  T  denotes  the  transpose  of  a  matrix.  It 

F  (x,  X)  =  f  (x)  +  cn  (x)X  (4) 

X  XX 

denotes  the  gradient  of  the  augmented  function,  the  optimum  solution  for  x  and  X 
must  satisfy  the  relations 

cKx)  =  0  ,  F  (x,  X)  =  0  (5) 

X 
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which  are  a  system  of  n  +  q  equations  in  the  n  +  q  components  of  x  and  X.  In 

Eqs.  (4)-(5),  the  gradients  f  and  F  denotes  n-vectors  and  the  matrix  cp  is 

x  x  x 

n  x  q. 

2.2.  Approximate  Solutions.  Since  the  system  (5)  is  generally  nonlinear, 
approximate  methods  must  be  employed.  In  this  connection,  we  introduce  here 
the  scalar  performance  indexes 


P(x)  =  cp  (x)cp(x) 


Q(x,  X)  =  fJ(x,  X)F  (x,  X) 

A  X 


(6) 


which  measure  the  errors  in  the  constraint  and  the  optimum  condition,  respectively. 
Then,  we  observe  that  P  =  0  and  Q  =  0  for  the  optimum  solution,  while  P  >  0  and/or 
Q  >0  for  any  approximation  to  the  solution.  When  approximate  methods  are  used,  they 
must  ultimately  lead  to  values  of  x  and  X  such  that 


P(x)  s;  ex  ,  Q(x,  X)  <:  e2 


(7) 


Alternatively,  (7)  can  be  replaced  by 


R(x,  X)  s: 


(8) 


where 


R(x,  X)  =  P(x)  +  Q(x,  X) 

denotes  the  cumulative  error  in  the  constraint  and  the  optimum  condition.  In 
(7)-(8),  Sj,  e^,  Sg  are  small,  preselected  numbers.  Note  that,  if  one 
chooses  =  e2  =  satisfaction  oilneq.  (8)  implies  satisfaction  of  Ineqs.  (7). 


urns. 
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3.  Description  of  the  Algorithms 

In  this  section,  the  algorithms  being  investigated  ure  described. 

(i)  SGRA-CR;  Sequential  gradient -restoration  algorithm,  complete 
restoration.  This  algorithm  consists  of  the  alternate  succession  of  gradient 
phases  and  restoration  phases. 

The  gradient  phase  is  started  providing 

P(x)  *  ex  (10) 

» 

It  involves  a  single  iteration,  in  which  the  augmented  function  is  reduced  subject 
to  an  upper  limit  for  the  constraint  error,  that  is,  ^ 

F(x,  X)  <  F(x,  X)  ,  P(x)se4  (11) 

The  restoration  phase  is  started  providing 

P(x)>ex  (12) 

It  involves  several  iterations,  in  each  pf  which  the  constraint  error  is  reduced, 
that  is, 

P(x)<P(x)  (13) 

The  restoration  phases  is  terminated  whenever  Ineq.  (10)  is  satisfied. 

^The  symbol  x  denotes  the  nominal  point,  x  the  varied  point,  and  X  the 
Lagrange  multiplier. 
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Remark .  The  algorithm  is  started  with  a  gradient  phase  if  Ineq.  (10) 
is  satisfied  or  a  restoration  phase  if  Ineq.  (10)  is  violated.  Normally,  a  gradient 
phase  is  followed  by  a  restoration  phase.  Occasionally,  the  gradient  phase  is 
followed  by  another  gradient  phase,  that  is,  the  restoration  phase  is  bypassed: 
this  is  precisely  the  case  whenever  Ineq.  (10)  is  satisfied. 

(ii)  SGRA  -IR:  Sequential  gradient -restoration  algorithm,  incomplete 
restoration.  This  algorithm  consists  of  the  alternate  succession  of  gradient 
phases  and  restoration  phases. 

The  gradient  phase  is  started  regardless  of  whether  Ineq.  (10)  is  satisfied. 

It  involves  a  single  iteration,  in  which  the  augmented  function  is  reduced  subject 
to  an  upper  limit  on  the  constraint  error,  that  is, 

F(x,  X)  <  F(x,  X)  ,  P(x)*P(x)+e4  (14) 

The  restoration  phase  is  started  only  if  Ineq.  (12)  is  satisfied.  It 
involves  a  single  iteration,  in  which  the  constraint  error  is  reduced  in  accordance 
with  Ineq.  (13), 

The  start' ug  condition  and  the  bypassing  condition  for  SGRA-IR  are 
identical  with  taose  of  SGRA-CR  (see  Remark). 

(iii)  SGRA -OR:  Sequential  gradient -restoration  algorithm,  optional 
restoration,  fnis  algorithm  consists  of  the  alternate  succession  of  gradient 
phases  and  restoration  phases. 

The  gradient  phase  is  started  providing 
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(see  Remark), 


AA*-*S 


(tv)  cam -NR:  Combined  gradteat-restaratim  algorithm. 


In  tins  algorithm,  the  gradient  phase  and  the  restoration  phase  are  confessed 
together  in  a  single  phase.  It  involves  a  single  iteration,  iaaMefetteapawcai 
function  Is  reduced  in  accordance  with  Ineqs.  (14). 


(v)  CGRA-AR:  Combined  gradient-restoration  algorithm. 


restoration.  This  algorithm  consists  of  the  alternate  succession  of  oomfetsed 
gradient-restoration  phases  and  restoration  phases. 

The  combined  gradient-restoration  phase  is  started  regardless  eS  whether 
Ineq.  (10)  is  satisfied.  It  involves  a  single  iteration,  in  which  the  avgmatfed 
function  is  reduced  in  accordance  with  Ineqs.  (14). 

The  restoration  phase  is  started  only  if  Ineq.  (12)  is  satisfied.  It 
involves  a  single  iteration,  in  which  the  constraint  error  is  redbred  m 
accordance  with  Ineq.  (13). 

The  starting  condition  and  the  bypassing  condition  for  CGSA-M  are 
identical  with  those  of  SGRA-CR  {see  Remark). 

(vi)  CGRA-OR:  Combined  gradient  -restoration  algorithm,  optional 
restoration.  This  algorithm  consists  of  the  alternate  succession  of  combined 
gradient-restoration  phases  and  restoration  phases. 

The  combined  gradient-restoration  phase  is  started  providing  Ineq.  (15) 
is  satisfied.  It  involves  a  single  iteration,in  which  the  augmented  function  is 
reduced  in  accordance  with  Ineqs.  (14). 

The  restoration  phase  is  started  providing  Ineq.  (18)  is  satisfied.  It 
involves  several  iterations,  in  each  of  which  the  constraint  error  is  reduced 
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is  ae*®nto«e  with  hcq.  (13).  The  restoration  phase  is  terminated  whenever 
&WR.  (J3)  is  satisfied. 

bj^passfeg  conditio*  for  CGRA-OR  is  identical  with  that  of  SGRA-CR 
(see  Remark). 

tesgt  For  the  algorithms  with  optional  restoration,  the  multiplier  X 
^ppearfqcia  (15HlS)is  computed  as  follows.  For  SGRA-OR,  Eq.  (19-1) 
nmst  ter  Mhed  wfth  Cj  =  1  aed  C2  =  0.  For  CGRA-OR,  Eq.  (19-1)  must  »*» 
whedaiAC,  =  1  anf  C-  =  1. 
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4.  Generalized  Algorithm 

Let  x  denote  the  nominal  point,  x  the  varied  point.  Ax  tl.~-  displacement 
leading  from  the  nominal  point  to  the  varied  point,  and  a  the  stepsize.  With 
this  understanding,  the  previous  algorithms  can  be  represented  in  the  following 
generalized  form: 


coJ(x)cpx(x)X  +  Cjcp^xiyx)  -  C^x)  =  0 

(19-1) 

p  =  Cjfx(x)  +  cpx(x)X 

(19-2) 

£ 

n 

4 

(19-3) 

x  -  x  +  Ax 

(19-4) 

For  given  nominal  point  x  and  constants  and  C ^  Eqs.  (19)  constitute  a 

complete  iteration  leading  to  the  varied  point  X  providing  one  specifies  the 

stepsize  a.  The  constants  C  and  C  depend  on  the  particular  algorithm  and 

1  *• 

take  the  values  given  in  Table  1.  The  detailed  derivation  of  Eqs.  (191  is 
presented  in  Refs.  1-3  and,  hence,  is  not  repeated  here. 
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Table  1.  Characteristic  constants. 


Algorithm 

Phase 

Ci 

C2 

Gradient 

1 

0 

SGRA 

Restoration 

0 

1 

Gradient  -restoration 

1 

1 

CGRA 

Restoration 

0 

1 

i. 


i  . 
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5.  Stepsize  Determination 

For  all  of  the  previous  algorithms,  the  position  vector  at  the  end  of  any 
step  can  be  written  as 

x  =  x  -  ctp 


(20) 


{  i 


where  p  denotes  the  search  direction,  which  is  given  by  (19-2).  This  is  a 
one-parameter  family  of  varied  points  x,  for  which  the  augmented  function  (3)  , 
the  constraint  error  (6 -IX  and  the  error  in  the  optimum  condition  (6-2)  take 
the  form 


L. 


i  . 


F(x,  X)  =  F(x  -  ctp,  X)  =  F(a) 


(21) 


P(x)  =  P(x  -  op)  =  P(a) 


Q(x,X)  =  Q(x  -ap,X)  =  Q(a) 


(22) 

(23) 


[  1 


For  the  gradient  phase  of  a  SGRA -algorithm  or  the  combined  gradient  - 
restoration  phase  of  a  CGRA  -algorithm,  Ineqs.  (11)  and  (14)  can  be  written 
in  the  general  form 

F(a)  <  F(0)  ,  F(<x)  *  P(0)  + 


U 


1 5 


(24) 


Their  satisfaction  can  be  ensured  by  employing  a  bisection  process,  starting 
from  a  suitably  chosen  reference  stepsize 


a  =  a 


ji 


? . 
<•  i 


(25) 
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L_ 

i 

!  . 
L. 

U 


y 


u 


u 


IL 


u 

t  I 

i  U.J 


I 


For  the  determination  of  the  reference  stepsize,  see  Section  6. 

For  the  restoration  phase  of  a  SGRA -algorithm  or  a  CGRA -algorithm, 

Ineq.  (13)  can  be  written  as 

P(a)  <P(0)  (26) 

Its  satisfaction  can  be  ensured  by  employing  a  bisection  process,  starting  from 
the  reference  stepsize 

a=  1  (27) 

This  value  reduces  the  constraint  error  P(x)  to  zero,  if  the  constraint  function 
ep(x)  is  linear  in  x. 


i 
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6.  Reference  Stepsize 

The  search  technique  outlined  in  Section  5  for  the  gradient  stepsize 
employs  a  bisection  process,  starting  from  the  reference  stepsize  (25), 
until  satisfaction  of  Ineqs.  (24)  occurs.  A  procedure  useful  to  determine  this 
reference  stepsize  is  outlined  here  and  is  based  on  a  quadratic  representation 
of  the  augmented  function  associated  with  the  one -parameter  family  of  solutions  (20). 

Let  the  function  F(a)  be  represented  in  the  quadratic  form 


I  ; 


F(a)  =  k0  +  kJa+  k2a 


(28) 


l  j 


and  let  the  coefficients  of  the  quadratic  be  determined  so  as  to  match  the  values 
of  the  ordinate  and  the  slope  at  a  =  0  and  the  value  of  the  ordinate  at  a=  1 . 

This  yields  the  relations 


,* 


i 

t  , 

i  . 


F(0)  =  kQ  ,  Fa(0)  =  k1  ,  F(l)  =  k0  +  k1  +  k2 


which  imply  that 


(29) 


'  t 

*  i 


kQ=F(0)  ,  kj  =  -Q(0)  ,  k2  =  F(l)-F(0)  +  Q(0) 


(30) 


With  the  coefficients  known,  the  following  possibilities  arise: 


(i)  k2  >  0  or  (ii)  k2  £  0 


In  Case  (i),  the  quadratic  function  (28)  has  a  minimum  for  the  following  value 
03  the  gradient  stepsize: 


(31) 


'  1 

Li 


'  1 


a=  -  k^/2k2 


(32) 
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In  Case  (ii),  the  quadratic  function  (28)  decreases  monotonically  with  a. 

This  suggests  the  use  of  the  following  reference  values  for  the  gradient  stepsize: 


°o=‘kl/2k2  “  k2>0 
a  =1  if  k„  £  0 


(33) 


18 


AAR -94 


[ 

7.  Experimental  Conditions  j 

In  order  to  evaluate  the  previous  algorithms,  eight  numerical  examples 
were  considered.  The  first  two  examples  pertain  to  quadratic  functions  subject  [ 

to  linear  constraints.  The  remaining  examples  pertain  to  nonquadratic  functions  j 

subject  to  nonlinear  constraints.  Each  example  was  solved  with  the  three 
versions  of  SGRA  and  the  three  versions  of  CGRA  outlined  in  Section  3.  All  of  j_ 

the  algorithms  were  programmed  in  FORTRAN  IV,  and  the  numerical  results  j 

i 

were  obtained  using  a  Burroughs  B-5500  computer  and  double -precision  arithmetic. 

i 

Starting  Point.  For  all  of  the  examples,  the  nominal  point  chosen  to  ! 

start  an  algorithm  was  defined  by 

Xi  =  x2=  ....  =xr=2  (34)  ; 

w.  sc  n  denotes  che  dimension  of  the  vector  x. 

Search  Technique.  The  determination  of  the  gradient  stepsize  and  the 
restoration  stepsize  was  performed  in  accordance  with  Sections  5  and  6.  For 
the  gradient  phase,  the  stepsize  a  was  subject  to  the  inequalities 

F(a)  <  F(0)  ,  P(a)  s  P(0)  + 1  (35)  . 

For  the  restoration  phase,  the  stepsize  was  subject  to  the  inequality 

P(a)  <  P(0)  (36) 

Convergence.  Convergence  of  an  algorithm  was  defined  through  the  inequalities 
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P(x)  £  10  8  ,  Q(x,  X)  ?  10  "4  (37) 

« 

Nonconvergence.  Conversely,  nonconvergence  of-ah)  algorithm  was 
defined  by  means  of  the  inequalities 

(a)  N  >  100  (38-1) 

or 

(b)  N  >20  (38-2) 

s 

or 

69 

(c)  M  >0.4x10  (38-3) 

Here,  N  is  the  iteration  number,  Ng  is  the  number  of  bisections  of  the  stepsize  a 
required  to  satisfy  Ineq.  (35)  or  (36),  and  M  is  the  modulus  of  any  of  the 
quantities  employed  in  the  algorithm.  Satisfaction  of  Ineq.  (38-1)  indicates 
divergence  or  extreme  slowness  of  convergence;  satisfaction  of  Ineq.  (38-2) 
indicates  extreme  smallness  of  the  displacement  Ax;  and  satisfaction  of  Ineq.  (38-3) 
indicates  exponential  overflow.  Each  of  these  situations  is  undesirable. 
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8.  Numerical  Examples 

In  this  section,  eight  numerical  examples  are  described.  The  first  two 
examples  pertain  to  quadratic  functions  subject  to  linear  constraints.  The 
remaining  examples  pertain  to  nonquadratic  functions  subject  to  nonlinear 
constraints. 

Example  8,1.  Consider  the  problem  of  minimizing  the  function 

f  *  (»1  -  x2)2  +  (x2  +  x3  -  2) 2  +  (x4  -  l)2  +  (x?  -  l)2  (39) 

subject  to  the  constraints 

Xi  +  3x2  =  0  ,  x3  +  x4-2x5=0  ,  x2-x5  =  0  (40) 

This  function  admits  the  relative  minimum  f  =  4.0930  at  the  point  defined  by 
x  =  -0.7674  ,  x  =  0.2558  ,  x  =  0.6279  ,  x  =  -0.1162  ,  x_  =  0.2558  (41) 

i  2.  6  i  3 

and 

Xj  =  2.0465  ,  X2  =  2.2325  ,  Xg  =  -5.9534  (42) 

Example  8,2.  Consider  the  problem  of  minimizing  the  function 

f  =  (4Xj  -  x2)2  +  (x2  +  x3  -  2) 2  +  (x4  -l)2  +  (x&  -  l)2  (43) 

subject  to  the  constraints 

xx  +  3x2  =  0  ,  x3+  x4  -  2xg  =  0  ,  x2  -  xg  =  0  (44) 

This  function  admits  the  relative  minimum  f  =  5,3266  at  the  point  defined  by 
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Xj  =  -0.9455  x  lO"1,  x2  =  0.3151  x  10“\  x3  =  0.5157,  =-0.4527,  x&  =  0.3151  x  10"J  (45) 


and 


X  =  3.2779  ,  X2  =  2.9054  ,  Xg  =  -7.7478  (46) 


Example  8.3.  Consider  the  problem  of  minimizing  the  function 


f  =  (Xj  -  l)2  +  (Xj  -  x2)2  +  (x2  -  x3)4 


(47) 


subject  to  the  constraint 


Xj(l  +  x2)  +  x4  -  4  -  3a/2  =  0 


(48) 


This  function  admits  the  relative  minimum  f  =  0. 3256  x  10  1  at  the  point  defined  by 


x  =1.1048  ,  x2=  1.1966  ,  x3  =  1.5352 


(49) 


and 


Xx  =  -0.1072  x  10 


-1 


(50) 


Example  8.4.  Consider  the  problem  of  minimizing  the  function 


f  =  (Xj  -  l)2  +  (x1  -  x2)2  +  (x3  -  l)2  +  (x4  -  l)4  +  (xg  -  l)6 


(51) 


subject  to  the  constraints 


2  4  2 

x^4  +  sin(x4  -  x&)  -  2J2  =  0  ,  x2  +  x^  -  8  -  J2  =  0 


(52) 


This  function  admits  the  relative  minimum  f  =  0.2415  at  the  point  defined  by 


■F 
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x  =  1.1661  ,  x  =1.1821  ,  x  =-  1.3802  ,  x  =  1.5060  ,  x  =  0.6109  (53) 

1  Z  ^  4  d 


Xx  =  -0.8553  x  lo"1  ,  X2  =  -0.3187  x  10"1  (54) 


Example  8.5.  Consider  the  problem  of  minimizing  the  function 


0 


f  =  (Xj  -  l)2  +  (x:  -  x2)2  +  (x2  -  x3)2  +  (x3  -  x4)4  +  (x4  -  xg)4  (55) 


subject  to  the  constraints 

xx  +  x2  +  x3  -  2  -  3V2  =  0  ,  x2  -  x2  +  x4  +  2  -  2J2  =  0  ,  x^  -2  =  0  (56) 

This  function  admits  the  relative  minimum  f  =  0.7877  x  10  1  at  the  point  defined  by 


x  =  1.1911  ,  x2  =  1.3626  ,  x3  =  1.4728  ,  x4  =  1.6350  ,  xg=  1.6790  (57) 


and 


X  =  -0.3882  x  10_1  ,  X2=  -0.1672  x  lo"1  ,  Xg  = -0. 2879  x  10“3  (58) 


Example  8.6.  Consider  the  problem  of  minimizing  the  function 


2  2  2 

f  =  0.01^  -  1)  +(x2  -  xp 


(59) 


subject  to  the  inequality  constraint 


ij 
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x1  +  x23  +  1  =  0  (61) 

Then,  the  previous  problem  can  be  recast  as  that  of  minimizing  the  function  (5^) 
subject  to  the  equality  constraint  (61).  The  function  (59)  admits  the  relative 
minimum  f  =  0. 04  at  the  point  defined  by 

Xi=-i  ,  x2=l  ,  x3  =  0  (62) 

and 


=  0.04  (63) 

Example  8.7.  Consider  the  problem  of  minimizing  the  function 


f=  “X. 


(64) 


subject  to  the  inequality  constraints 


VX1  ’ 


2 

VX1 


(65) 


Introduce  the  auxiliary  variables  x  and  x  defined  by 

O  *x 


3  2  n  2  2  A 

X2  "  X1  "  X3  =  0  *  XI  "  x2  "  X4  =  0 


(66) 


Then,  the  previous  problem  can  be  recast  as  that  of  minimizing  the  function  (64) 
subject  to  the  equality  constraints  (66).  The  function  (64)  admits  the  relative 
minimum  f  =  -1  at  the  point  defined  by 


X  =  1  ,  x„  =  1  ,  X-  =  0  ,  x,  =  0 


24 


and 

.  ^*-1  m 

Example  S.8.  Consider  the  problem  ofiafejmia^fte  fmtaim 

f=iogx3-%2  m, 

subject  to  the  equality  constraint 

x*  +  x*-4  =  0  Ctt 

and  die  inequality  constraint 

*3^1  CFU 

Introduce  the  auxiliary  variable  defined  by 

2 

x3=l  +  *1  0*1 

The,:>  the  previous  problem  can  be  recast  as  that  of  minim  iziag  the  function 

2 

f  =  log(l  +  x1) -x2  03) 

subject  to  the  equality  constraint 

2  2  2 

(1  +  x2)  +  x2  -  4  =  0 

Note  that  has  been  eliminated  from  the  problem  and  can  be  computed  a  posteriori 
with  (72).  The  function  (73)  admits  the  relative  minimum  f=-J3at  the  point  defined  by 
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9.  Results  and  Conclusions 

The  examples  described  in  Section  8  were  solved  with  the  three  versions  of 
SGRA  and  the  three  versions  of  CGRA  described  in  Section  3.  The  numerical 
results  are  presented  in  Tables  2-3,  where  the  number  of  iterations  for 
convergence  is  shown.  For  the  eight  examples  considered.  Table  4  shows 
the  cumulative  number  of  iterations  for  convergences  £N*.  From  die  tables, 
the  following  conclusions  arise:  (a)  a  restoration  of  some  form  is  necessary  for 
rapid  convergence;  and  (b)  while  SGRA-CR  is  the  most  stable  among  the 
algorithms  considered  here,  rapidity  of  convergence  can  be  increased  somewhat  if 
one  employs  algorithms  with  alternate  restoration  or  optional  restoration. 


Table  2.  Number  of  iterations  for  convergence  N# 


Example 

SGRA-CR 

SGRA-IR 

8.1 

5 

5 

8.2 

8 

8 

8.3 

18 

14 

8.4 

56 

51 

8.5 

8 

7 

8.6 

15 

12 

8.7 

9 

15 

8.8 

11 

11 

Table  3.  Number  of  iterations  for  convergence  N 


Example  CGRA-NR  CGRA-AR 
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